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SPATIAL VIDEO PROCESSING 



This invention relates to video processing and particularly to spatial 
video processing. An important example is the creation of additional lines 
through spatial interpolation. 

A spatial interpolation filter has a filter aperture comprising an 
appropriate number and arrangement of neighbouring pixels and takes a 
weighted sum of contributions from pixels within that filter aperture. It has 
previously been recognised that such a filter can introduce smearing if there 
are pronounced slopes in the picture material. It is instructive to take the 
example of a striped image. A filter designed to take weighted sums of 
contributions along a horizontal pixel line will be expected to function correctly 
as long as the stripes in the image are horizontal. Essentially, weighted 
averages are taken along the stripe and therefore from pixels which are 
similar. If diagonal stripes are encountered, a weighted sum of contributions 
along a horizontal pixel line will now mix pixels which are from different stripes 
and which may be very different. Smearing of the image is likely to result. 

One approach to this problem is to avoid using spatial interpolation and 
instead to utilise temporal interpolation. This might work well for stationary 
pictures but is unlikely to be a satisfactory solution where there is movement 
between successive pictures. 

It has been proposed to improve spatial interpolation by measuring 
pronounced slopes in the picture material and then rotating the filter aperture 
into alignment with the measured slope. In the example of diagonal stripes, 
the angle of the stripe is measured and the filter aperture rotated so that 
weighted contributions are taken along the stripe, from pixels that remain 
similar. This proposal can be very helpful if the slope is measured accurately 
and the filter rotated into precise alignment. This can however be a complex 
operation. Moreover, if the filter is rotated by the wrong amount, the results 
can be as bad as, or sometimes worse than, if the aperture remained static. It 
is also a factor that if the filter aperture is to be able to accommodate shallow 
slopes (and it is shallow slopes that produce the most objectionable artefacts), 
a very wide filter aperture is required. 



It Is an object of this invention to provide improved video prooessing in ' 
which slopes are accommodated with greater tolerance to errors or vanat,ons 
in slope angles. 

It is an object of one form of this invention to meet the performance of 
prior art slope processing with substantially smaller filter apertures. 

Accordingly, the present invention consists in one aspect in video 
processing apparatus comprising a set of spatial filter apertures and a slope 
detector the apparatus selecting the appropriate filter aperture in accordance 
with the output of the slope detector and taking weighted contributions from 
pixels in the selected filter aperture, wherein the filter aperture weightings sum 
to unity over a line including the current pixel and sum to zero over either s.de 

of the said line. 

Suitably, said line is vertical. 

Advantageously, the set consists of three spatial filter apertures. 

in another aspect, the present invention consists in video process.ng 
apparatus comprising a slope detector and a spatial filter having a pos.t.ve 
f„ter aperture, a linear filter aperture and a negative filter aperture; where.n 
the positive filter aperture is employed upon detection of any positive slope ,n 
excess of a defined positive threshold; the negative filter aperture is employed 
upon detection of any negative slope in excess of a defined negafive 
threshold; and the linear filter aperture is employed otherwise. 

Advantageously, in each said filter aperture weighted contributions are 
taken from pixels with the filter aperture weightings summing to unity over a 
line including the current pixel and summing to zero over either s.de of the 

said line. 

Suitably, said line is vertical. 

The invention will now be described by way of example with reference 
to the accompanying drawings, in which- 

Figure 1 is a diagram showing the pixel weightings for a known (.near 

f " ter * Figure 2 is a diagram showing the pixel weightings for a positive slope 
filter aperture according to an embodiment of the invention; 

0 



Figure 3 is a diagram showing the pixel weightings for a negative slope 
filter aperture according to an embodiment; 

Figure 4 is a diagram illustrating the manner of operation of the 

described embodiment; and 

Figures 5 to 8 are diagrams illustrating an adaption process according 

to an embodiment of the invention 

The example will be taken of vertical interpolation. If a four tap vertical 
filter - such as that shown in Figure 1- is used to vertically interpolate an 
interlaced picture the resulting image will be soft with various artefacts. One 
of the most noticeable artefacts is jagging on diagonal lines. Attempts have 
been made to try and reduce this jagging by twisting the vertical filter so it 
filters along the direction of the slope of the diagonal line. This will remove 
the jags, but such a filter is very dependent on knowing the exact slope of line, 
which is difficult to measure. 

A preferred embodiment of the present invention uses a set of three 
filters. Rather than twisting the simple vertical filter of Figure 1 to interpolate 
along the direction of the slope, a switch is made to the filter of Figure 2 for a 
positive slope and to the filter of Figure 3 for a negative slope. (The terms, 
positive slope and negative slope, in a picture, are often found confusing. In 
this specification, the nomenclature that is used denotes as positive a slope 
which rises vertically, from left to right.) 

It can be seen that the filters of Figures 2 and 3 use high frequency 
contributions from horizontally displaced pixels. Thus weightings of the 
contributions from the central vertical line are the same as in the simple 
vertical filter and sum to unity. The weightings on either side of that line, sum 
to zero. High frequency information from adjacent pixels is thus being used to 

reduce jags in the image. 

A possible explanation for the excellent results achieved with this 
arrangement is that it allows the removal of jags from edges that are very 
nearly horizontal (the most objectionable), using a fairly small aperture. 
Twisting the aperture to interpolate along the edge would require a much 
larger aperture. 
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Referring to Figure 4, consider the picture contains a sioping edge, 
denoted by the solid vertica, »ne. The voltage across the edge vanes as 
snown by .he marKed parabola. I. a vertical ,0.5 0.5, AUer we. used me 
resulting interpolated voltage would be (b + f)/2. which ,s considerably larger 

he correct result, d. if the aperture were sKewed along the drage a^e 
es .ting voltage would be (c*e,/2. In order for this to give the correct result 
I aperture would have to be 16 pixels wide horizontally. However, usrng the 
l P e Alter according to this embodiment of .he invention, the interpolafed 
voltaae is <b+f-a+c-g-re)/2, which gives the correct answer. 
, one embodiment, the adaption or switch between the positive, linear 

and negative slope AUer apertures Is effected as follows. The positive and 
negative gradients of the input picture signal are Ars« ™ 
difference of these two values is then calculated, giving a gradten, difference 
or switching signai. This signal is then Altered horizontal,, and vertically «o 
5 produce a sort of threshold signal. This Altering step a,so >™**^ 
aperture size, considering pixels other than the particular ones ■!"<*«* 

a'ysed in fhe s,ope Alters, thereby ensuring that the adapAon treats a b d 
range of slopes. If the threshold signal is within a specified range, he usual 
Tear Alter is used. However, i, I, is above this range, the positive slope Alter ,s 
,n used and similarly, if it is below, the negative Alter is preferred. 

' ,n more detail, the adaption process includes the following steps:- 

. Calculate the positive gradient (g,) using the Alter shown in Figure 5 

25 . calculate the negative gradient <g m ) using the Alter shown in Figure 6 

. Calculate the difference (d) of the absolute value of these two signals 
d = ( abs(g p ) - abs(g m ) ) 

30 o Filter using 0.25 0.5 0.25 vertically and 0.25 0.5 0.25 horizontal to 
give fd 
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• Threshold signal f d to form positive slope switching signal for f d > 5, 
negative slope switching signal for f d < -5 and linear switching signal for 
-5 < fd < 5 

It is advantageous that the gradient filters detect a range of slopes, 
5 which is why they are filtered vertical and horizontally by the 0.25 0.5 0.25 
filters. Although this increases the aperture size of the mode selection filters 
(see Figure 7) it produces a worthwhile improvement in picture quality. 

The above adaption process is equivalent to dividing the the gradient 
space into the areas shown in Figure 8. It will be recognised that this is only 
1 o one example of a thresholding arrangement and modifications will be 
possible. 

Slope orientation filtering has been shown to give a significant 
improvement on picture quality over linear filtering. It does not appear to 
introduce significant artefacts and operates to reduce jagging without 

1 5 softening the picture. Although the described slope filters appear to work very 
well, alternatives are possible. In certain applications it may be appropriate to 
have more than one positive slope filter and more than one negative slope 
filter, with each filter aperture still having the feature that the pixel weightings 
sum to unity over the central line of the aperture and sum to zero over all 

20 pixels on each side either side of the central line. 

Although the example has been taken of vertical interpolation, this 
invention encompasses other spatial filter operations and orientations other 
than vertical. In, for example, a horizontal interpolation filter according to this 
invention, pixel weightings would sum to unity over the central horizontal line 

25 of the aperture and sum to zero over all pixels on either side of the central 
line. 

Similarly, although the adaption signal described here behaves 
reasonably well, numerous alternatives are possible. It will often be useful to 
include a temporal filter aperture, for use in place of the described spatial 
30 apertures, where there is no detected motion. 



Some applications may require both de-interlacing and temporal 
interpolation. If this is the case there are a number of possible methods. 

1 . De-interlace first, followed by temporal interpolation - the 
disadvantage with this is the need to store the 1 st field. 

2. De-interlace both fields at once and then temporally interpolate. 

3 include the temporal interpolation into each of the three adaption 
filters, in this case you would use the same adaption control for 
each temporal phase. 
,t should be understood that this invention has been described by way of 
examples only and that a wide variety of further modifications are poss.ble 
without departing from the scope of the invention. 
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CLAIMS 



1 . Video processing apparatus comprising a set of spatial filter apertures 
and a slope detector, the apparatus selecting the appropriate filter 
aperture in accordance with the output of the slope detector and taking 
weighted contributions from pixels in the selected filter aperture, 
wherein the filter aperture weightings sum to unity over a line including 
the current pixel and sum to zero over either side of the said line. 

2. Apparatus according to Claim 1 , wherein said line is vertical. 

3. Apparatus according to Claim 1 or Claim 2, wherein the set consists of 
three spatial filter apertures. 

4. Video processing apparatus comprising a slope detector and a spatial 
filter having a positive filter aperture, a linear filter aperture and a 
negative filter aperture; wherein the positive filter aperture is employed 
upon detection of any positive slope in excess of a defined positive 
threshold; the negative filter aperture is employed upon detection of 
any negative slope in excess of a defined negative threshold; and the 
linear filter aperture is employed otherwise. 

5. Apparatus according to Claim 4, wherein in each said filter aperture 
weighted contributions are taken from pixels with the filter aperture 
weightings summing to unity over a line including the current pixel and 
summing to zero over either side of the said line. 



6. Apparatus according to Claim 5, wherein said line is vertical. 
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Figure 1: Linear aperture 
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Figure 2: Aperture for positive slope 
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Figure 3: Aperture for negative slope 
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Figure 4: 
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Figure 5: Gradient to detect positive slope 
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Figure 6: Gradient to detect negative slope 
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Figure 7: Aperture size for gradients 




Figure 8: Adaption control 
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